Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dataset): use sqlglot for DML check #31024

Merged
merged 2 commits into from
Nov 22, 2024
Merged

fix(dataset): use sqlglot for DML check #31024

merged 2 commits into from
Nov 22, 2024

Conversation

betodealmeida
Copy link
Member

SUMMARY

Use sqlglot when getting column metadata for a virtual dataset. We currently use sqlglot to check for DML in SQL Lab, but when we save a query as a dataset we're still using sqlparse.

Part of SIP-117.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

Run this query in SQL Lab:

with source as ( select 1 as one ) select * from source

This query is flagged incorrectly as DML by sqlparse, but not by sqlglot.

After running, save the query as a dataset. It should work, but currently it fails because sqlparse flags the query as having DML.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@dosubot dosubot bot added the data:dataset Related to dataset configurations label Nov 21, 2024
@betodealmeida betodealmeida merged commit 832fed1 into master Nov 22, 2024
33 checks passed
asher-lab added a commit to asher-lab/superset that referenced this pull request Nov 22, 2024
* fix: Exception handling for SQL Lab views (apache#30897)

* fix: don't show metadata for embedded dashboards (apache#30875)

* feat: add logging durations for screenshot async service (apache#30884)

* refactor(Avatar): Migrate Avatar to Ant Design 5 (apache#30740)

* refactor(input): Migrate Input component to Ant Design 5 (apache#30730)

* fix(empty dashboards): Allow downloading a screenshot of an empty dashboard (apache#30767)

Co-authored-by: Geido <[email protected]>

* chore(ci): add tai and michael to helm owners (apache#30925)

* fix(helm): use submodule on helm release action (apache#30924)

* docs: add Free2Move to INTHEWILD.md (apache#30930)

* fix: blocks UI elements on right side (apache#30886)

Co-authored-by: Evan Rusackas <[email protected]>

* chore(deps): Migrate from `crate[sqlalchemy]` to `sqlalchemy-cratedb` (apache#29243)

* chore: update change log, UPDATING.md and bug-report.yml for 4.1 release (apache#30915)

* chore(docs): Update list of supported databases to include CrateDB (apache#30946)

* feat(trino,presto): add missing time grains (apache#30926)

* fix(Dashboard): Exclude edit param in async screenshot (apache#30962)

* fix(Card): Use correct class names for Ant Design 5 Card component (apache#30964)

* chore(Accessibility): Fix accessibility for 'Show x entries' dropdown in tables (apache#30818)

* build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /superset-frontend/cypress-base (apache#30969)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(release validation): scripts now support RSA and EDDSA keys. (apache#30967)

* build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /docs (apache#30970)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: add performance information to tooltip (apache#30948)

* fix(helm-chart): Fix broken PodDisruptionBudget due to introduction of extraLabels. (apache#30966)

* chore(helm): bumping app version to 4.1.1 in helm chart (apache#30918)

* chore: add unit tests for `is_mutating()` (apache#31021)

* chore(🦾): bump python numexpr 2.10.0 -> 2.10.1 (apache#31006)

Co-authored-by: GitHub Action <[email protected]>

* chore(🦾): bump python cffi 1.16.0 -> 1.17.1 (apache#31002)

Co-authored-by: GitHub Action <[email protected]>

* chore(🦾): bump python flask-babel subpackage(s) (apache#31000)

Co-authored-by: GitHub Action <[email protected]>

* fix(Dashboard): Retain colors when color scheme not set (apache#30646)

* fix(explore): verified props is not updated (apache#31008)

* chore: publish wheels (apache#30981)

* docs: Embedded sdk (apache#30972)

* fix(imports): import query_context for imports with charts (apache#30887)

* docs: Update doc about CSV upload (apache#30867)

Co-authored-by: Evan Rusackas <[email protected]>

* chore(🦾): bump python cattrs 23.2.3 -> 24.1.2 (apache#30998)

Co-authored-by: GitHub Action <[email protected]>

* fix(dataset): use sqlglot for DML check (apache#31024)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Michael S. Molina <[email protected]>
Co-authored-by: Joe Li <[email protected]>
Co-authored-by: Maxime Beauchemin <[email protected]>
Co-authored-by: Mehmet Salih Yavuz <[email protected]>
Co-authored-by: Geido <[email protected]>
Co-authored-by: Ville Brofeldt <[email protected]>
Co-authored-by: Paolo Terzi <[email protected]>
Co-authored-by: Sukuna <[email protected]>
Co-authored-by: Evan Rusackas <[email protected]>
Co-authored-by: Andreas Motl <[email protected]>
Co-authored-by: Andreas Motl <[email protected]>
Co-authored-by: Levis Mbote <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <[email protected]>
Co-authored-by: Christoph Keller <[email protected]>
Co-authored-by: lodu <[email protected]>
Co-authored-by: Beto Dealmeida <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Action <[email protected]>
Co-authored-by: JUST.in DO IT <[email protected]>
Co-authored-by: David Hotham <[email protected]>
Co-authored-by: Giampaolo Capelli <[email protected]>
Co-authored-by: Linden <[email protected]>
Co-authored-by: Seiya <[email protected]>
Co-authored-by: Asher Manangan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data:dataset Related to dataset configurations size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants